The present invention relates generally to image capturing systems, and, more particularly, to increasing the dynamic range of image sensing devices, such as CMOS pixel circuits, having limited dynamic range.
Digital cameras and other electronic image capturing systems have become increasingly common. In such systems, photographic film is replaced by an array of photo sensors. Although such systems have a number of advantages with respect to film based system, they typically lack the dynamic range of photographic film.
A common type of image sensing element uses CMOS active picture elements. Such image sensor devices of the CMOS active picture element variety rely on having an image sensing area divided into a large number of photosensitive areas, or pixels. These areas produce an output signal determined by the light incident on the physical location of the pixel, with the pixel information being read out to a set of external electronics following a predetermined scanning sequence. Random addressing, sequential addressing, or some combination of the two may be used. A typical pixel circuit using three transistors is shown in FIG. 1.
In FIG. 1, the diode D1 is normally biased in its reverse direction, so that if it is previously charged to a voltage, photons impinging on the diode will produce carriers which discharge its terminal voltage as seen at wire 3. Transistor M1 serves as a reset transistor for recharging the diode voltage. When the row voltage VRES on wire 1 is taken in a sufficiently positive direction, it causes conduction in transistor M1 and charges the diode voltage towards the voltage VDD on wire 4. This reset action is normally initiated at the start of a time when the diode will be used to accumulate photon information about the impinging light's intensity.
After a predetermined exposure time, the light intensity information for this pixel is read out by using the transistors M2 and M3. Transistor M2 serves as a source follower, with the voltage on its source at 5 being a function of the diode voltage at 3. The voltage source VDD on wire 4 provides a current to operate the source follower. When the particular row of pixels containing this pixel is selected for readout, the row voltage VSEL on wire 2 is taken in a positive direction, turning on transistor M3. Transistor M3 is usually operated as a switch, to connect the source terminal of M2 to the readout column line 6. A current sink I at the end of the column line provides operating current for the source follower M2, so that the output voltage on the column line VOUT will be proportional to the voltage on the diode at 3.
After the intensity information is read out and is no longer needed, the row reset input VRES may be activated to cause the pixel voltages to be restored to the value representing zero light intensity. In addition to functioning as a reset transistor, M1 may also be used to limit the lower voltage to which wire 3 may descend. This prevents diode D1 from becoming zero or forward biased, and therefore prevents the injection of minority charges into the substrate. Charges injected into the substrate may diffuse to adjacent pixels, causing spreading or blooming in the image of bright image spots. In many systems, this anti-blooming feature is used to hold the pixel voltages at their lower level until the reset action is initiated to prepare for a new image exposure.
For simple image sensor usage, the pixel in FIG. 1 may be formed into an array of cells with control and readout circuitry surrounding it on a single silicon integrated circuit. FIG. 2 shows an exemplary two-dimensional image sensor conceptual diagram.
In the two-dimensional imager of FIG. 2, a rectangular area 10 is populated with an array of pixels such as those shown in FIG. 1, with some of the row and column wires shown for clarity. Control logic 11, primarily located at an edge of the pixel array, operates the VSEL and VRES wires for each row of the pixel array. Counters, gates, and/or shift registers in this logic generate the control signals needed to follow the desired pixel readout sequence. When a pixel row is selected, the information from it goes to the readout circuitry 12, as signals on the columns wires 13. The resulting pixel information is output on 14 as either analog or digital signals as needed.
For normal sequential scanning, the control logic 11 activates one row reset signal VRES at a time, following a linear scan pattern descending from the top of the pixel array 10 to the bottom. Consider for this example that at the time of interest, the VRES signal is being applied to row A. At the same time, the control logic also sends the select signal VSEL to a different row, which we may choose to be row B in this example. The pixel information from row B will then be sent to the readout circuitry 12 on the entire set of column wires, of which 13 is an example, and the readout circuitry will choose which column information to send out on the connection 14.
The total pixel exposure interval for a row of image sensor pixels is determined by the delay time between the application of a first VRES and a second VRES to a particular pixel row. During this time period the light illuminating the image sensor is not interrupted. The amount of sensor pixel row exposure per interrogation cycle interval is determined by the delay time between the application of the VRES and the VSEL to a particular sensor pixel row. If rows are being scanned in a regular periodic manner, this time interval will be determined in turn by the row number spacing between row A and row B. Thus the amount of time the image sensor is exposed to light will be a function of the row timing delay between the row reset action and the row readout action as determined by the control logic. Image readout from a different row, such as row C with a different relative location, will give different effective image exposure time.
Although this discussion is based on one of the simpler three transistor cells in use, this configuration exhibits the sort of limited dynamic range due to saturation that is also found in the various other image sensing element designs known in the art. Compared to the dynamic range available from film, such elements will consequently exhibit saturation when exposed to a bright image while still being under exposed for a dark image.
There are numerous methods and apparatus that have been proposed to increase the dynamic range of photo sensors in general, and CMOS sensors in particular. Perhaps the most straightforward is just to increase the amount of charge that is stored by the diode D1 of FIG. 1; however, this results in a larger pixel circuit, when the trend is to reduce pixel size in order to increase resolution.
The prior art presents a number of alternate techniques for dealing with these problems; however, these suffer from one or more serious practical problems, such as requiring nonstandard sensors, increased memory requirements, or multiple sensor resets. For example, U.S. Pat. No. 6,977,685 is illustrative of the class of CMOS sensor dynamic range enhancement technologies that rely upon resetting individual pixels during the course of photo sensor exposure to light as the primary means of effecting dynamic range extension. In U.S. Pat. No. 6,946,635 the sensor pixels are read out two or more times without resetting the image sensor, but require that these readings are separated by a shutter opening and closing cycle at the point of sensor (photo detector) re-exposure, so that the exposure of the array is not continuous for a single image. U.S. Pat. No. 6,115,065 employs a specific, specialized image sensor that incorporates a memory cell with each image sensor pixel and calls for resetting and reading each sensor pixel site twice, one over a short exposure interval and one over a long exposure interval, and clocking out both of these values from the photo sensor. According to the teachings of U.S. Pat. No. 6,018,365, the dynamic range of an imaging system that utilizes an array of active pixel sensor cells is increased by reading each cell in the array multiple times during each integration period, saving the number of photons collected by the cell, resetting the cell if it would normally saturate by the end of the integration period, and summing of the values collected during the integration period, an approach requiring both the resetting of cells and additional memory. United States Patent Application 20020067415 offers a method of operating a solid state image sensor having an image sensing array that includes a plurality of active pixels, the resetting of each pixel, and after successive time periods, reading outputs from each pixel, to obtain multiple sets of image data having different dynamic ranges that are then combined, an approach again requiring both the resetting of cells and additional memory.
All these various approaches suffer from the need to either use non-standard CMOS sensors that are significantly more costly to manufacture and control than the CMOS sensors commonly in use, or require significantly more image buffer memory to properly operate. One proposed approach for increasing CMOS sensor dynamic range, while using standard CMOS sensors, is to capture two to more images, taken very close together in time, at different exposures, at least one which displays the shadow detail of the image (taken at a high exposure) and at least one which displays the highlight detail (taken at a low exposure). These multiple images, which need to be corrected for any motion present in the images occurring between individual image capture, are combined into a single image that displays both shadow and highlight detail. This approach does result in an increase of dynamic range; however, the processing complexity to motion compensate and appropriately combine the two or more captured images, and the additional buffer memory required, is excessive by current standards, and therefore this solution, to date, has not been shown as a practical answer to the problem. The need to use specialized, non-standard sensors, add memory, or reset pixels during the expose interval all have serious drawbacks. Consequently, there is a need for improved techniques to increase the dynamic range of photo sensors.